|
|
|
| e-Pub |
Previous | 
Home
Section: New Results
Experimentation, Emulation, Reproducible Research
This section covers our work on experimentation on testbeds (mainly Grid'5000), on emulation (mainly on Distem), and on Reproducible Research.
Grid'5000 design and evolutions
Participants : Jérémie Gaidamour, Arthur Garnier, Lucas Nussbaum [contact] , Clément Parisot.
The team was again heavily involved in the evolutions and the governance of the Grid'5000 testbed.
In the context of ADT LAPLACE, Jérémie Gaidamour adapted and configured the CiGri middleware on Grid'5000. CiGri enables the execution of large campaigns of best-effort jobs (low priority, interruptible jobs). It is expected that this work will allow the remaining free time slots to be filled by tasks from other research communities such as natural language processing.
Jérémie Gaidamour also greatly improved stats5k, our tool to generate metrics about the testbed (usage, resources availability, etc.), available at https://intranet.grid5000.fr/stats/ .
Arthur Garnier added the testing of Grid'5000 tutorials to our continuous integration installation, enabling the earlier detection of problems on the testbed. He then led the migration to PostgreSQL as the backend for the OAR batch scheduler – a behind-the-scenes but major migration.
In addition to daily administratrive duties and to his work on Kwapi described below (section 7.3.2 ), Clément Parisot added support for production workloads to Grid'5000, extending the scope of the testbed to make it more suitable for additional user communities. He then managed the installation of the new clusters at Nancy, purchased in the context of OIP Grid'5000 and CPER CyberEntreprises.
Finally, in addition to his roles in the bureau, comité d'architectes and comité des responsables de sites of Grid'5000, Lucas Nussbaum managed the purchase of the new clusters at Nancy mentioned above, and gave several presentations about the testbed, at Journées SUCCES [14] , at Retour d'expéRiences sur la Recherche Reproductible [15] , and at École Cumulo Numbio.
A unified monitoring framework for energy consumption and network traffic
Participants : Lucas Nussbaum [contact] , Clément Parisot.
Providing experimenters with deep insight about the effects of their experiments is a central feature of testbeds, that Grid'5000 was only partially addressing. We designed Kwapi, a framework that unifies measurements for both energy consumption and network traffic. Because all measurements are taken at the infrastructure level (using sensors in power and network equipment), using this framework has no dependencies on the experiments themselves. Initially designed for OpenStack infrastructures, the Kwapi framework allows monitoring and reporting of energy consumption of distributed platforms. In this work, we extended Kwapi to network monitoring, and overcame several challenges: scaling to a testbed as large as Grid'5000 while still providing high-frequency measurements; providing long-term loss-less storage of measurements; handling operational issues when deploying such a tool on a real infrastructure.
This work was published at Tridentcom [31] and presented in a GENI/FIRE collaboration workshop [12] . It is now in production as the default monitoring framework on Grid'5000.
Comparison of HPC and Clouds testbeds
Participant : Lucas Nussbaum [contact] .
Given the recent launch of two large NSF-funded projects that share similar goals as Grid'5000 (CloudLab and ChameleonCloud), we worked on analyzing the design choices made so far by those projects, comparing them with Grid'5000. Preliminary results were presented at REPPAR [17] and at a GENI/FIRE collaboration workshop [13] .
Emulation with Distem
Participants : Emmanuel Jeanvoine, Lucas Nussbaum [contact] , Cristian Ruiz.
Several improvements have been made around Distem, mostly in the context of ADT COSETTE.
During the internship of Arthur Carcano, we tried to use Distem to experiment on NDN infrastructures. We obtained promising results, especially in terms of scale. We plan to continue this work and publish it in 2016.
We also submitted, to CCGRID, a paper demonstrating the use of Distem to evaluate fault tolerance and load balancing strategies implemented in Charm++. This submission is still pending evaluation.
Finally, in an effort to validate Distem performance, we studied the performance of Container-based virtualization technologies such as LXC or Docker, as most of the underlying technology is also shared with Distem. We studied their performance in the context of HPC, and showed that containers technology has matured over the years, and that performance issues are being solved. This work has been published at VHPC [43] .
Management of large-scale experiments
Participants : Emmanuel Jeanvoine, Lucas Nussbaum [contact] , Cristian Ruiz.
Following our survey of experiment management tools [7] accepted at FGCS at the end of 2014 and published early this year, we worked on Ruby-Cute, a library that aggregates various useful functionality in the context of such tools. We hope that it will be useful as a basis for future tools, and ease testing of new ideas in that field. The library is available on http://ruby-cute.github.io/ .
Tracking provenance in experiment control tools
Participants : Tomasz Buchert, Lucas Nussbaum [contact] .
In the context of our work on XPFlow, we worked on the collection of provenance during experiments. We surveyed provenance collection in various domains of computer science, introduced a new classification of provenance types suited to distributed systems experiments, and proposed a design of a provenance system inspired by this classification. This work has been published at REPPAR [29] .
Reproducible Research
Participant : Lucas Nussbaum [contact] .
Lucas Nussbaum gave a presentation on Reproducible Research[16] at the ICube laboratory seminar (Strasbourg). A shorter version of the talk was given to the Inria Comité des projets in Nancy.
Lucas Nussbaum also co-organized the second edition of REPPAR, a workshop on Reproducibility in Parallel Computing, held in conjunction with Euro-Par'2015.